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prediction interval provides an expected upper and lower bound on balance load pre- 
diction at a given confidence level. A method has been developed which accounts for 
sources of variability due to calibration and check-load application. The prediction 
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idation of the methods is demonstrated for the case study based on the probability of 
capture of confirmation points. 
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I. Introduction 


A wind-tunnel test program often involves a complicated model installation process which may 
include installation of hardware that bridges the metric to non-metric gap. Clearances between 
non-metric supports and the balance supported test article may involve tight tolerances such that 
support deflections under aerodynamic loading or large environmental changes that may lead to 
contact. Even in the case of a simple installation, it is always prudent to apply known forces and 
moments to a model/balance and observe that these loads are being correctly measured prior to 
testing - a check-loading [1]. Current practices for confirmation of check-loads are typically informal 
and involve using the standard deviation of the calibration residuals with a coverage factor based 
on the desired confidence. A more mathematically defensible bound on the prediction of loads may 
be provided by a prediction interval, which contains two general sources of uncertainty. The first 
is that due to the balance uncertainty obtained during the balance calibration. The second is the 
uncertainty in the applied loads created by the check-load hardware. 

II. Estimates of Uncertainty in Balance Responses 

Estimates of uncertainty in load measurement from the balance are obtained during the calibra- 
tion process. Estimates include the uncertainty in setting the applied loads due to the calibration 
hardware used, as well as the error associated with the balance reading of the loads through the 
calibration model. Set-point errors as well as any measurement system error are also included in 
these estimates. A rigorous, statistical approach to the calibration process provides an estimate 
for the variance in each of the component bridge voltage responses through the Mean Square for 
Error (MSE) available from the Analysis of Variance (ANOVA). The developed regression model 
depends on the design point load combinations, expressed here in coded units (a:). A typical model 
used with monolithic moment balances contains at least primary sensitivities, 1st order interactions, 

two-factor interactions, and pure quadratics as shown in Eq. (1) [1]. 

6 6 6 

Uk = A) + ^2 P iXi + X! X! ftij x i x j + ^ + e (1) 

*= 1 j = 1 i<j *= 1 

The response is the bridge voltage of the component of interest, y. The x’s in the model represent 
the applied loads given in coded factors. Each estimated term in the regression model (/?,;) is 
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represented as a column in the calibration model matrix X used during the regression calculations. 
The calibration model matrix X contains only statistically significant terms, tested at the desired 
significance level (a). One challenge faced in using calibration models is that the user ultimately 
requires the output in force and moment engineering units, not voltages. The calibration model is 
y = f (x) whereas the desired output is x = / (■ y ). 

A prediction interval is a bound on the predicted response for a given applied loading - for 
instance a confirmation point during calibration. The vector Xo is used to define a single multi- 
component load combination, expanded to include all terms in the calibration regression model. 
The t-statistic is computed for a given level of confidence (1 — a), and (n — p) residual degrees of 
freedom. There are n unique calibration design points (or load combinations) in the calibration 
design and p parameters in the resulting regression model. The prediction interval for a single 
confirmation point evaluated during calibration may then be expressed as Eq. (2) [2, 3]. 

y (x 0 ) ± t a/2}n - p ^/ MSE (l + x' (X'X)" 1 x 0 ) (2) 

This prediction interval for response voltage can then be expressed as an upper and lower bound 
for forces and moments by computing the loads through the balance calibration model. This model 
is self-consistent for use during calibration, but the bias error (expressed as a variance) associated 
with the uncertainty in applied loads due to the calibration hardware (fr^ ias _ ca i) must be added to the 
random component (MSE) for the overall uncertainty [4, 5]. Note that the bias must be expressed 
as a voltage here for unit consistency. 


V (*0) ± *a/2,„- p y ( MSE + ^bias-cal) (l + *0 (X'X) _1 X 0 ) (3) 

III. Uncertainty in Loads Applied by the Check-Load Hardware 

The check-loading hardware is used to apply known total loads to the balance/model including 
the loads due to the hardware. The mass properties of the check- load hardware and all dimensions 
that define the moment arms should be measured with the highest possible precision. A software 
program can then be developed for incorporating these precise measurements to calculate the loads 


3 



applied to the test article by the check- load hardware (including tare weights). A method is now 
required to combine all the individual uncertainties. A Monte Carlo simulation allows integration 
of all individual uncertainties including all dimension tolerances, and the uncertainty in the applied 
and tare weights [4]. An estimate of the standard deviation is required for each source of uncertainty. 
The Monte Carlo simulation models each error source as a normal distribution using the mean value 
of the parameter and standard deviations. Overall standard deviations in each response are then 
calculated for each of the runs to provide an absolute bias estimate expressed in units of response 
voltage for the applied loads (^ ias . applied ). Details are shown in the case study that follows. 

IV. Check-Load Prediction Interval for Total Uncertainty 

The prediction interval used for check-loading expressed as a balance bridge output is given as 
Eq. (4). It is based on the development presented in Reference 3 but includes the additional bias 
due to the applied loads via the check-load hardware and the bias from the calibration [5] . 

V (Xo) ± t a/2 ,n- p] j (^bias-applied + MSE + a^) (l + x' (X'X)" 1 X 0 ) (4) 

This interval now allows the user to choose a load combination, apply it through the check-load 
hardware, read the balance response and determine if this response captures the applied load in 
the prediction interval. The balance response is a voltage and many may feel that it is easier to 
interpret a response in the engineering units for a force and moment. The calibration matrix may be 
used to back calculate the force and moment values in engineering units using the method outlined 
in Reference 1. 


V. Case Study: The NASA Langley In-Situ Load System 

Specialized hardware has recently been developed at NASA Langley Research Center (LaRC) 
to provide rapid, in-situ validation of wind-tunnel model balance loads in the wind tunnel just prior 
to testing. The case study presented here coupled this new hardware with a typical balance used 
in the NASA LaRC National Transonic Facility (NTF). The study involved loading the balance 
using a calibration fixture on the metric end while supporting the balance on a system capable of 
adjusting both pitch and roll orientation on the non-metric end. 
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A. The ILS Concept 


The In-Situ Load System (ILS) is a new system designed at NASA LaRC to help address the 
issues surrounding a system-level validation or calibration of a wind-tunnel model system (WTMS) 
[6]. Together, the aircraft model, model sting, balance, angle measurement system, and other 
instrumentation make up the WTMS. The ILS incorporates the rigorous methodology described 
above for quantifying uncertainties to standardize the check-loading process. The ILS concept has 
origins in the NASA LaRC Single- Vector Calibration System (SVS), which exploits the use of a 
single deadweight loading to create variable, multi-component loads through rotation and offset of 
the point of load application with respect to the balance moment center (BMC) [7]. Fig. 1 shows 
the ILS and illustrates the principles of operation. The ILS has a two degree-of-freedom joint which 



Fig. 1 The ILS Mounted to a Balance Calibration Fixture 


allows the applied load to remain aligned with the gravitational vector. The upper bearing mount 
supports two bearings which in turn support the bearing cross shaft. The bearing cross contains a 
third bearing which is tied to the lower bearing mount. Loads are applied through a weight hanger 
attached to the lower bearing mount. For the purpose of this study, the ILS was attached directly 
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to a balance calibration fixture. Offsets in the x- and y-direction are afforded through multiple 
mounting holes in the ILS mount and the fixture (axes shown in Fig. 2). The balance is inserted 
in the fixture block and supported and oriented on the non-metric end by allowing movement in 
both the pitch and roll axes. An Angle Measurement System (AMS) is installed to the front of the 
calibration fixture to provide the orientation of the balance with respect to the gravitational vector. 
The AMS package consists of three Q-flex accelerometers that are oriented orthogonally [8] . 

B. Loads Applied by the ILS 

The design of the ILS hardware ensures that the load vector coincides with the gravity vector. 
Based on the balance coordinate system, the load vector is resolved into the three forces. F f )Q ; is 
the vector of the three forces defined as [F x F y F z ]' . 

Fbal = F a pp g (5) 

where F app is the magnitude of the applied load vector, and g is the gravity vector describing the 
orientation of the balance expressed as [g x g y g z ]' . For a constant F app , the magnitude of the three 
forces is varied by changing the orientation of the balance in pitch and roll. The components of the 
gravity vector are expressed in terms of c/’s. The moments applied to the balance are not only a 

Non-Metric End 



Fig. 2 Balance Moment Center Reference Axes 

function of the applied load vector and the orientation of the balance, but also the distance of the 
load point from the BMC. The BMC is an imaginary point on the balance that is used to define 
the balance coordinate frame, and is the point by which all forces and moments are referenced. 
The distance from the BMC is a vector expressed as d BMC = [x BMC y B uc z bmc }' , where the sign 


6 



convention is defined in Fig. 2. The applied moments about BMC, Mf, a {, are the cross product of 
the distance vector and the balance force vector, or 

Mbal — d BMC X F bal (6) 

where M ba i is the vector of the three moments defined as [M x M y M z ]' and F ba i is given by Eq. 5. 


C. The NTF-113C Balance 

The NTF-113C balance is a single-piece, six-component moment balance designed for full-span 
testing at the NTF. Table 1 shows the full-scale design loads for the NTF-113C. The NTF-113C 


Table 1 NTF-113C Balance Design Loads 


Balance 

Component 

Aerodynamic 

Component 

Design Load 

English Units, lbs. or in-lbs. Metric Units, N or N-m 

F z 

NF 

6,520 


29,002 

F x 

AF 

400 


1,779 

My 

PM 

12,800 


1,446 

M x 

RM 

8,150 


920 

M z 

YM 

6,400 


723 

Fy 

SF 

4,000 


17,792 


was selected for this study since the ILS has a maximum applied load limit of 5,000 lbs., which is 77 
percent of the full-scale capacity of normal force. Full-scale loads are possible with the remaining five 
components using the ILS. The bridge electrical configuration is such that the strain-gage bridges 
are in a force-balance format (NF1/NF2, SF1/SF2, RM, AF), where the beam elements are located 
in two cages equally spaced at axial stations forward and aft of the BMC as shown in the overview 
drawing of Fig. 3 

D. The ILS Check-Load Study 

The load schedule shown in Table 2 was used in testing the ILS hardware with the NTF-113C 
balance. The applied load was fixed at 2,500 lbs for the testing since a lower value of F app allowed 
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Fig. 3 NTF-113C Balance Overview 


for a larger volume of the six-dimensional space to be explored. By increasing F app , the design 


Table 2 ILS Check-Load Schedule for NTF-113C 


Point No. 

Load Point 

9x 

9y 

9z 

F x , 

1 yi 

F z , M x , My, 

M z , 

M appi 






lbs. 

lbs. 

lbs. in-lbs. in-lbs. 

in-lbs. 

lbs. 

1* 

1 

0.028 

0.249 

0.968 

70 

623 

2420 -6 -4833 

1244 

2500 

2 

1 

0.131 

0.395 

0.909 

328 

988 

2273 -3531 -2244 

1485 

2500 

3 

2 

0.131 

0.395 

0.909 

328 

988 

2273 -3531 2870 

-738 

2500 

4 

3 

-0.113 

0.332 

0.937 

-283 

830 

2343 -1992 2796 

-1232 

2500 

5* 

4 

0.057 

-0.136 

0.989 

143 

-340 

2473 2975 -4313 

-765 

2500 

6 

4 

0.087 

0.130 

0.988 

218 

325 

2470 -2844 -3650 

731 

2500 

7 

5 

-0.070 

-0.122 

0.990 

-175 

-305 

2475 2669 -1531 

0 

2500 

8* 

5 

0.000 

0.000 

1.000 

0 

0 

2500 0 0 

0 

2500 

9 

6 

-0.122 

0.138 

0.983 

-305 

345 

2458 -3019 2862 

-776 

2500 

10* 

6 

-0.057 

0.000 

0.998 

-143 

0 

2495 0 4363 

0 

2500 

11 

7 

0.113 

-0.332 

0.937 

283 

-830 

2343 1992 -2796 

-1232 

2500 

12 

8 

0.131 

-0.395 

0.909 

328 

-988 

2273 3528 2870 

737 

2500 

13 

9 

-0.113 

-0.332 

0.937 

-283 

-830 

2343 1992 2796 

1232 

2500 


*Note: Asterisk indicates a load combination that is replicated for estimate of experimental error 

limits of the balance would have been exceeded at the same orientations given in Table 2. The load 
schedule included all nine load points and primarily focused on complex, multi-component loads. 


Simple combinations, such as a single-component force, provided a baseline to compare the ILS 
hardware to readily available hardware, like a knife-edge weight hanger. Replicates of some load 
combinations were performed to estimate the pure experimental error [2]. Additionally, the load 
schedule was replicated over several days to assess any day-to-day variability. 

VI. Determination of Prediction Intervals 
A. Uncertainty in the NTF-113C Balance Responses 

The first and typically dominant component for estimating balance calibration uncertainties is 
the residual error from the mathematical model, expressed as MSE. The NTF-113C is calibrated 
using the NASA LaRC SVS and features a modified Central Composite Design (CCD) for specifying 
the load schedule. This classic design is perhaps the most popular design for estimating a second- 
order response model in many fields. The excellent prediction variance properties and efficient run 
schedule have been the subject of much discussion in the literature [2, 3]. The mean square for 
error is an overall variance estimate resulting from the analysis of variance (ANOVA) and is given 
in the calibration report for a balance. Values for MSE and residual degrees of freedom (df) for all 
components of the the NTF-113C are given in Table 3. Bias error associated with the uncertainty 

Table 3 NTF-113C Calibration Model Residual Statistics 



F x 

Fy 

F z 

M x 

M y M z 

Residual df ( n — p) 

45 

49 

50 

52 

55 54 

MSE (fiV/V) 

2.680 0.245 

0.198 0.485 0.997 0.762 


in applied loads due to the calibration hardware must be added to MSE for the overall uncertainty. 
The uncertainty in loads applied by the SVS has been described in Reference 5. Uncertainties 
in loads applied by the ILS to the NTF-113C balance were computed using this method and are 
presented in Table 4. 
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Table 4 Estimated Bias Errors due to Calibration with SVS 


F x Fy F z M x My M z 
Avg Variance (lbs.; in-lbs.) 2 0.3750 0.3803 0.3386 8.0715 7.5861 18.0000 

Avg Std Dev (lbs.; in-lbs.) 0.6124 0.6167 0.5819 2.8410 2.7543 4.2430 

Max Std Dev (lbs.; in-lbs.) 0.6833 0.7326 0.6692 4.3688 4.5272 4.7060 

Min Std Dev (lbs.; in-lbs.) 0.0817 0.0815 0.0872 0.0832 0.0581 0.2852 


B. Uncertainty in Loads Applied by the ILS 

The weight of the ILS hardware was not subtracted by a tare sequence, but rather included in 
the calculation of the desired applied load. This requires precise knowledge of the weight and center 
of gravity of the ILS hardware, which were experimentally determined for both the ILS Mount 
and remaining ILS subassembly. The mount and the ILS subassembly are moved independently 
to generate load combinations so that their mass properties were determined separately. The ILS 
Subassembly consists of the Upper Bearing Mount and the Bearing Cross as seen in Fig. 1. The 
details of the ILS geometry and measured mass properties may be found in Reference 6. It was 
assumed that the force due to the lower bearing mount acts through the load point and therefore 
the location of the center of gravity is not required. The additional applied force to the balance 
is the summation of the deadweight load and the weight of the ILS hardware. In addition to 
precisely determining the mass properties, all critical dimensions that define the moment arms from 
BMC were evaluated using a calibrated coordinate measuring machine (CMM). The uncertainties 
associated with all involved measurements are summarized in Table 5. 

A Monte-Carlo simulation was developed to combine all the uncertainties, including the toler- 
ance of the CMM-based measurements, uncertainty in the mass of the applied and tare weights, and 
the uncertainty in the AMS measurements of the gravity vector. The software program incorpo- 
rated the precise measurements, perturbed with an assumed normal distribution using the standard 
deviation estimates of Table 5, to calculate the forces at the BMC including the tare weight applied 
to the balance fixture by the ILS. When using the ILS, uncertainty in the forces is primarily a 
function of the uncertainty in determining the g-vector. The uncertainty in the applied moments is 


10 



Table 5 Elemental Error Sources for the In-Situ Load System 


Symbol 

Description 

Standard Deviation 

Units 

u 9x 

Projection of Gravity vector on *-axis 

0.0001 

9’* 

u ay 

Projection of Gravity vector on y - axis 

0.0001 

9’* 

u gz 

Projection of Gravity vector on 2 -axis 

0.0001 

9’* 

UxBMC’ Uy BMC > U ZBMC 

CMM Linear Measurements 

0.0002 

in. 

UCG X 

Center of Gravity Measurement on m-axis 

0.0014 

in. 

UcGy 

Center of Gravity Measurement on j/-axis 

0.0025 

in. 

UCG Z 

Center of Gravity Measurement on 2 -axis 

0.0011 

in. 

Ur w 

Force due to Precision Weights 

0.01% F.S. 

lbs. 


Force due to ILS Components < 5 lbs. 

0.00022 

lbs. 


Force due to ILS Components > 5 lbs. 

0.0022 

lbs. 

a function of the moment 

arm lengths in addition to the forces and 

as a result, additional variability 

is introduced. A pseudo-code for the Monte Carlo approach for one 
is given below with comments. 

design point (load combination) 


Begin 

For i=l to 1000 iterations 

U Uncertainty added to Gravity Vector, nrand computes a random value from a normal 
U distribution using a standard deviation of Ug and mean of zero. 

G= [gx , gy , gz] +nrand (Ug) 

U Uncertainty added to distances from BMC to ILS Mount (mt) and ILS subassembly (sub) 
dBMCmt= [CGx , CGy ,CGz]mt + [nrand(UCGx,UCGy,UCGz)]mt + [xBMC ,yBMC ,zBMC]mt 
+ [nrand(Ux,Uy,Uz)]mt 
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dBMCsub= [CGx , CGy ,CGz]sub + [nrand(UCGx,UCGy,UCGz)l sub + [xBMC ,yBMC ,zBMC]sub 
+ [nrand(Ux,Uy,Uz)]sub 

U Uncertainty added to Forces of Mount and Subassembly, Force at Balance Calculated 
U (bal) 

Fmt= [Wmt+nrand(UFs) ] G 

Fsub= [Wsub + nrand(UFs or UF1)]G 

Fbal= Fmt+Fsub 

U Moments at BMC Calculated 
Mmt = dBMCmt X Fmt 
Msub = dBMCsub X Fsub 
Mbal= Mmt + Msub 

7o"/o Store Mbal and Fbal in array 

If i > 1000 go to * 

If i < 1000 begin next iteration i=i+l 

H ^Calculate the mean and standard deviation of each component force and moment using 
U the 1000 trials 


End 


Squaring the overall standard deviation ( a ) in each response calculated for each of the runs provides 
an absolute bias estimate for the ILS applied loads, 0b ias _ lied . E &c h orientation of the ILS pro- 
vides a different load combination (design point) and requires a separate Monte Carlo simulation. 
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Results from this study are summarized in Table 6 with upper and lower bounds from different load 
combinations. The most accurate value for cr dias _ applied is found by using the computed value at 
the given orientation. An engineering compromise may be to look at the bias over all orientations 
tested and take the average. This was the approach taken in this case study using the orientations 
of Table 2; average values are given in Table 6. Conversion from calculated forces and moments 

Table 6 Bias in ILS Applied Loads over Study Design Points 

F x Fy /•'. M x My M z 
Avg Variance (lbs.; in-lbs.) 2 0.0627 0.3918 0.1184 5.5590 0.0653 5.3490 

Avg Std Dev (lbs.; in-lbs.) 0.2503 0.6259 0.3441 2.3580 0.2556 2.3130 

Max Std Dev (lbs.; in-lbs.) 0.2676 0.8999 0.3720 2.7400 0.2834 2.5480 

Min Std Dev (lbs.; in-lbs.) 0.2293 0.0006 0.3152 2.1380 0.2341 2.1390 


in engineering units to response voltages is then done using balance primary sensitivities. Table 6 
provides a summary of computed bias values averaged over the complete range of ILS orientations 
for this study. The average variance (first row of Table 6) was used for cr dias . applied in subsequent 
prediction interval calculations. 

C. Prediction Interval Capture Probability for the ILS Case Study 

All terms in the prediction interval of Eq. (4) are now known. The user may choose to convert 
the response voltages back to units of force and moment for ease of interpretation. Use of balance 
primary sensitivities for this conversion represents a reasonable engineering compromise. 

The load combinations given in Table 2 were executed and replicated over four days on the 
NTF-113C balance. Responses from the six strain-gage bridges were recorded and the six balance 
loads were estimated using the iterative balance reduction process [1]. The most recent calibration 
matrix for the NTF-113C balance was used to estimate the balance loads. Once the six balance 
loads were estimated, prediction intervals based on Eq. (4) were applied to the estimated loads. The 
intervals were compared with the physics-bases calculations of the applied load to check whether 
the applied load fell within the six prediction intervals. The acceptable error rate, a, in Eq. (4) was 
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set to 0.05, which yielded a 95-percent confidence level for a single prediction interval. The overall 


probability that the six prediction intervals simultaneously captured the known physics-based load 
was at least 1 — (6 x 0.05) = 0.70 or 70 percent. The Bonferroni adjustment to the error rate was 
made to account for the simultaneous nature of the prediction intervals [9]. For an overall capture 
probability of 95 percent, the new error rate, a*, based on Bonferroni’s method was set to a* = a/6 
and a* replaced a in Eq. (4). Table 7 summarizes the capture probabilities from the ILS check- load 


Table 7 Summary of Capture Probabilities from Check-Load Study 


95% Prediction Interval 2a Calibration 

Date Total Number of Number of Captured Percent Number of Captured Percent 


Combinations 

Points 

Captured 

Points 

Captured 

07/11/2013 

156 

150 

96.2 

142 

91.0 

07/12/2013 

240 

236 

98.3 

212 

88.3 

07/15/2013 

126 

119 

94.4 

113 

89.7 

07/23/2013 

126 

121 

96.0 

106 

84.1 

Total 

648 

626 

96.6 

573 

88.4 


testing. A total of 108 load combinations (108 x 6 = 648 points) were performed over the four-day 
period. The last column of Table 7 compares the capture rate for the two standard deviations of 
the back-computed residuals to the capture rate from the newly-derived prediction interval method. 
This comparison is made due to the perceived popularity of using the former interval for check- 
loading. The overall capture rate using the back-computed 2a method was 88.4 percent versus 
the prediction interval presented here, which captured 96.6 percent of the check-load points. As 
mentioned previously, the capture rate was expected to be at least 95 percent. 

Table 8 shows a representative single check-load and the comparison between the actual applied 
load and the estimated load from the six balance bridges. The load combination in the example is 
point number 12 in Table 2. It is worth noting that applying the exact combination as stated in 
Table 2 is not important since an AMS measures the orientation of the balance, and the applied 
forces and moments are easily calculated from the physics-based equations given by Eqs. (5) and 
(6). In this example, the actual load for five of the six components falls within the prediction 
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Table 8 Example Comparison of Actual and Estimated Loads with Prediction Intervals 


Component 

Estimated 

Load, 

lbs. or in- lbs. 

Actual 

Load, 

lbs. or in-lbs. 

95% Prediction 
Interval Half-Width, 
lbs. or in- lbs. 

2cr Calibration 
Interval Half- Width, 
lbs. or in-lbs. 

F x 

324.60 

325.10 

2.90 

1.32 

Fy 2 

-986.39 

-982.55 

3.88 

2.40 

F z 2 

2277.47 

2275.00 

4.71 

2.10 

M x 

3497.50 

3497.93 

28.94 

20.44 

My 

2817.59 

2821.24 

28.63 

14.18 

M z 1,2 

721.83 

742.39 

19.44 

9.00 


*Note: A 1 indicates an applied load outside of the 95% prediction interval and a 2 indicates an applied 

load outside of the 2a calibration interval 

intervals on the estimated load. It is noted that the prediction intervals for the moments are larger 
due to additional uncertainties in the moment arm lengths. Table 8 also provides a comparison of 
the prediction interval half-widths of this study and half-width intervals computed using two times 
the standard deviation of the back-calculated residuals [1], Using the 2cr calibration intervals, only 
three of the six components were captured. 


VII. Conclusion 

The development of the ILS hardware naturally led to the need for a rigorous estimate of the 
prediction interval for a given combined loading. Accounting for all of the contributing sources of 
variation for a prediction interval calculation represents a formidable accounting task. The honest 
assessment of the newly-derived prediction interval provided a capture rate for loadings tested of 
over 96 percent. The use of bias values derived from averages over a range of the applied load 
combinations used during calibration and check-loading may prove adequate based on these initial 
results. Future work may address the potential differences in prediction interval point capture 
afforded through detailed bias assessments at each individual load combination. In addition, the 
method should be proven with other balance and check-load hardware combinations. 
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